A LEVEL REDUCTION THEORY OF INDEFINITE 
BINARY QUADRATIC FORMS 

CARLOS CASTANO-BERNARD 

Abstract. In this paper we study a geometric coding algorithm 
for indefinite binary quadratic forms Q for the congruence sub- 
group T'^{N), with respect to the usual fundamental domain 
where A'' is assumed prime. The cycles Qi, . . . , Qn that this algo- 
rithm produces are such that the the corresponding paths 71 , . . . , 7„ 
in the Riemann surface Xq{N){<C) have a nice behaviour around 
the elliptic points of order 2. 
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1. Introduction 

In this paper we study a geometric coding algorithm for indefinite 
binary quadratic forms Q for the congruence subgroup r'^(A^), with 
respect to the usual fundamental domain jFjy, where is assumed 
prime. (Cf. Schoeneberg book [8].) Following Ogg [6] let us call E- 
points the elhptic points of order 2 with respect to the action of V^{N) 
on the upper half-plane 

H = {r G C: ^{t) > 0}. 

The cycle 

Ql 1 • • • 1 Qn 

that this algorithm attaches to Q deals with the i?-points is such that 
the the corresponding paths 
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define in a natural way a regular path on the Riemann surface Xq{N){C). 

The motivation to write this paper was mostly utilitarian. In a forth- 
coming paper of the author this algorithm is applied to the computation 
of the homology class of each of the connected components of the real 
locus of the Atkin-Lehner quotient Xq{N) of Xq{N) associated to the 
Pricke involution w^. 

2. The level 1 case 

2.1. Preliminaries. Let [A, B,C]{X,Y) = AX^+BXY+CY^ denote 
a binary quadratic forms with real coefficients A, B, and C. Note that 
for any real numbers a, /3, 7 and 6 we have 

2aP a6 + /3-f 2-f6\ \ b\ . 
(3^ P5 J \Cj 

Let M = {'^D e GL2(M) and define 

Let Q denote the set of binary quadratic forms. Note that 

{Q,Q') = BB' -2{AC' + J^C). 

makes Q a quadratic space and M i— > um defines a group homomor- 
phism 

a : GL2(M) 0+(Q) 

of the general linear group GL2(M) into the group of orientation pre- 
serving automorphisms 0'^{Qi) of the ternary quadratic form iQp = 
{Q,Q) on Q} For each matrix M G GL2(]R) and each binary qua- 
dratic form Q E Q we write M o Q = gmQ- The ternary quadratic 
form IQp = {Q,Q) is called the discriminant of the binary quadratic 
form Q — [A, B, C\. We say Q is definite (resp. indefinite) if iQp > 
(resp. IQP < 0). If Q is definite and A > then we say Q is positive 
definite. 

Definition 2.1. Two binary quadratic forms Qi and Q2 are called 
equivalent if there exists M e SL2(Z) such that M o Qi — Q2. 

Example 2.2. Any form Q — [A, B,C] is equivalent to 

SoQ^[C,-B,A], 
where ,5= (o-qI) e SL2(Z). 

-^In fact the homomorphism (Tm restricted to SL2 (K) = {to G GL2(R): detw = 
±1} is a degree 2 cover of SL2(K) known as the spin representation attached to the 
ternary quadratic form = {Q, Q). 
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Example 2.3. Any form Q = [A, B,C] is equivalent to 

T* o Q = [A, -2tA + B, t^A - tB + C], 
where T = ( } * ) e SL2(Z), for each t e Z. 

Definition 2.4. We say a positive definite binary quadratic form Q — 

[A, B, C] is normal if -A< -B < A. 

Every positive definite form Q is equivalent to a normal form, namely 

A-B 



2A 

and [x\ denotes the floor of a real number x. The form Q""" is known 
as the normalisation of Q. 

Definition 2.5. We say a positive definite binary quadratic form Q — 
[A,B,C] is reduced if Q is normal and if any of the following two 
conditions is true. 

(1) A<C 

{2) A = C and -B <0 

Algorithm 1: The reduction of a positive definite form Q 
Data: any positive definite binary quadratic form Q 
Result: a reduced form Q'^^ equivalent to Q 

1 Q ^ Q"'^™; 

2 while not reduced do 

3 Q^iSoQ)-^^; 

4 end 



The definition of reduced form may be rephrased in geometric terms 
as follows. To each positive definite form Q attach the point ttq & H 
defined by 

-B + \Q\ 

where \Q\ denotes the square root of \Q\'^ such Q'(IQI) > 0. So Q is 
reduced if and only if ttq & T', where 

J^' =i^Ten: - ^ < K(r) < ^|n{r eU: \t\ > 1 or (|t| = 1 and 3?(r) < 

which is depicted by Figure 2.1. If we define r i— > Mr for each M — 
(" ^) e SL2(]R) by the Mobius transformation 

ar + P 



7T + 

then we have an action of SL2(R) on H such that Mttq — ttmoQ- 
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• oo 




Figure 2.1. The fundamental set T' . 

While performing Algorithm 1 on a given positive definite form Q 
we may obtain a product of matrices 

Rq^T^'S--- ST^'^ST''^ e SL2(Z) 

such that Rq o Q is reduced, where S = and T = {W)- Note 

that in terms of the above geometric setting we have Rqtvq G J-'. So 
given any t E 7i there is a matrix Rr G SL2(Z) such that RrT G J-. 
In fact Rr is uniquely determined by r, except when r is equivalent 
to either i — -\/^, with stabihser of order 2 or p_i = -i^V^ ^ ^j^j^ 
stabihser of order 3. 

Remark 2.6. Later, in connection with the coding algorithm we will 

find convenient to use the closure jF/v = (JF')*^'" of the fundamental set 
JF^. In the literature the closed set T gH is known as a fundamental 
region. 

2.2. Indefinite forms. Let sign{x) denote the sign of a real number 
X. We will find convenient to assume the somewhat non-standard con- 
vention that sign{0) = 1, as opposed to the usual sign{0) = 0. 

Definition 2.7. We say an indefinite binary quadratic form Q — 
[A, B,C] is reduced^ if S > and if any of the following two conditions 
is satisfied. 

Rl: 2\A + C\ < B 

R2: 2{A + C) = -sign{A)B 



A closely related definition of reduced indefinite form may be found in Choie and 
Parson's article [3], where Q = [A,B,C\ is called reduced if A > 0, C > 0, and 
A + C <B. 
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From now on let us assume Q = [A,B,C] is an indefinite binary 
quadratic form with A, B and C in Z. We shall give a geometric inter- 
pretation of the concept of reduced indefinite form. Provide the upper 
half-plane Ti with the hyperbolic metric 

y 

where r = x + iy, with x and y the usual (real-valued) coordinate 
functions of C. Let {ti,T2} denote the (oriented) geodesic that joins a 
given point Ti G 7Y* to a given point T2 & H*. The geodesic 7q attached 
to an indefinite form Q is 

f {^.^} if-"^o 

7q = < {ioo, -§} if A = and S > 

1^ {-§,ioo} ifA = OandS<0 

where \Q\ denotes the positive square root of We also write 

a/D = \Q\, where D = — AAC. The intersection number 1(71,72) 
of given (oriented) geodesies arcs 71 and 72 on the upper half-plane H.* 
is defined by 

{1, if #(71 n 72) = 1 and {li{P)il2{P)} has positive orientation 
—1, if #(71 n 72) = 1 and {7i(-P), 72(-P)} has negative orientation 
0, otherwise. 

where ^'{P) denotes the unit tangent vector of a geodesic 7 at a point 
P e 7. 

Example 2.8. Suppose Q = [0, 1,0] and U = [1,0, —1]. In fact 7q = 
{ioo, 0} and = !}• Clearly and 7^7 meet at P = 7r[i^o,i] = i 
(and no other point). In particular 7(7q,7[/) = ±1. To determine the 
sign we note that the tangent vector of 7q at P is given by 7q(P) = 
(0, —1), and the tangent vector of 7^7 at P is given by 7[/(P) = (IjO). 
Since the determinant 

-1 

1 



> 



we have I{^q,^u) = 1- 



Put a — {p_i,p+i}, where as before p±i — — 2 - ^o^® a C ^u, 
where U — [1,0,-1]. To simplify our exposition assume from now on 
that 



D is not a perfect square. 



Lemma 2.9. An indefinite form Q is reduced if and only ifI{'jQ, a) = 1 
and 7q n J^'"* ^ 0. 

Proof. First note that given two binary quadratic forms Qi and Q2, 
where Qi is positive definite and Q2 is indefinite, the condition that 
the point ttq^ lies on the geodesic 7Q2 is equivalent to {Qi, Q2) — 0. So 
7q n 7c7 7^ means there is a positive definite form Q' such that 
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(1) {Q',U)=0 

(2) {Q',Q)^0 

Write Q' = [A', B', C']. Clearly (1) and (2) hold if and only if 

A'^C and B'B -2{AC' + A'C) ^0. 

This means that B'B - 2{AA' + A'C) = 0. In other words 

B' _A + C 
2A' ~ B 

Thus 7q meets at a point ttq/ on the geodesic arc a — {p_i, p+i} if 
and only if 

2|A + C| < \B\. 

Now we split the proof in two cases according as ttq/ is an endpoint of 

a or not. 

Case I: Suppose 

2(A + C) 

This condition is equivalent to p±i G 7q. So 7q fl 7^ if and only 
if 

B 

and /(7(5, u) = 1 if and only if ±A > 0. In other words, 2{A + C) — 
-sign{A)B and B > 0. 
Case II: Now suppose 

2\A + C\ < \B\. 

Since the point ttq/ is not an endpoint of a, we have 'Jq fl JF*"* 7^ 0. 
Moreover, the condition S > is equivalent to I{'')Q,a) = 1. The 
lemma follows. □ 

We will shortly introduce Algorithm 2,^ which is a reduction algo- 
rithm of indefinite binary quadratic forms that we will use in connection 
with Algorithm 3. The latter algorithm is a variant on the geometric 
coding algorithm of Katok [4]. 

We call an indefinite form Q = [A,B,C] nearly reduced if 3A^ < D, 
where D = B^ — 4 AC is the discriminant oi Q. To each nearly reduced 
indefinite form Q we attach the interval 



Jc 



^-.i5n(A),Ci5n(A)] > if D < AA'' 



where 



,1 , B + VD- 3^2 1 B-VD- 3A2 
'^^ = ^2 + 2A ' '^^-^2 + 2A ■ 



^The author owes the main ideas underlying this algorithm to a talk given by 
Richard Pinch. 
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Note that the assumption that Q is nearly reduced guarantees that the 
end points of the interval Jq are real. 

Lemma 2.10. Suppose Q is a nearly reduced indefinite form. We have 
I(lTtoQ, a) ^1 if and only ifte Jq, where T"* = ( J f ). 

Proof. First we define an auxiliary function fg attached to the nearly 
reduced form Q — [A,B,C] by letting 



where 

At = 
Bt = 
Ct = 

with t G M. Note that 7^*0 
geodesic arc a 



A. 

B - 2At, 
Ae -Bt + C, 

has non-trivial intersection with the 
is such that 



p+i} if and only if ^ e 

/qW < I- 

We will see that the set of such t may be expressed as a union of two 
closed intervals. Our first step will be to find the endpoints of these 
intervals, i.e. the solutions t G M of = |. The latter is equivalent 
to any of the following equations 

2\A + Ct\ = \Btl 

2\At^ -Bt + A + C\ = \B- 2At\, 

2{At^ -Bt + A + C) = ±{B - 2At). 

A straight forward calculation shows that the four real numbers t+i 

and defined above are all the solutions of fqit) 
the proof splits in two cases. 
Case I: Suppose D > AA^. This condition means that the intervals 
t+] and [t'_, t'_,_] arc disjoint. It is straight forward to see that fQ{t) < 
|, if and only if t G U [t'_,t'j_\, with fqit) = | exactly at the 

endpoints. Clearly /(7rtoQ,cr) = 1 if and only if 

te ^1,^+1]. 

Case II: Suppose D < AA"^. This condition means that the intervals 
[t-, t'_] and [t+, t'_^] are disjoint. Again it is straight forward to see that 
fqit) < i if and only if t G [t-,t'_\ U with fq{t) = | exactly 



i. Now 



at the endpoints. Note that /(7TtoQ,c) 
(2.1) 



P—sign{A) 



B 



1 if and only if 



2\A\ 



By squaring the left-hand side of Inequahty 2.1 we get 

2 



P-sign(A) t + 



B 
2A 



P-sign(A) t + 



B 
2A 



P-sign{A) t + 



2A, 



CARLOS CASTANO-BERNARD 

B\ ^ B 



( - \ ( B\ ^ B 

P-sign{A)P-sign{A) + yP-sign{A) + P-sign{A) ) {^^2^}^^ ~ li^^ 



4A2 



• ..^/ B\ , B B^ 

sign{A){-t+—)+f--t + 



2AJ A AA^ 

So Inequality 2.1 is equivalent to 

2 f . B\ , ^^B B^ D 

e + [stgn{A) -^jt- stgn{A)— + - + 1 < 0. 

After a straight forward calculation we may see that t e R satisfies 
Inequality 2.1 if and only if f e t-sign{A)-i't'_sign{A) ' ^^'^ lemma 
follows. □ 

Corollary 2.11. If Q = [A, B, C] is an indefinite form such that fl 
T ^ ^, then there is exactly one t e Z such that o Q is reduced, 
unless D < AA? and either p_i or p+i lies in 7q. 

Proof. Wc split the proof sphts in two CclSGS clS follows. 

Case I Suppose Q = [A, B, C] is an indefinite form such that D > AA^. 

Note that the closed interval Jq has length 1. Now we consider two 

sub-cases. 

Sub-ceise I: There is exactly one point t e Jq n Z. Note that 
7t*oq n .F'"* 7^ 0. Thus by Lemma 2.10 we may see T* o Q is 

reduced. 

Sub-case II: The endpoints t'_^ and t'j^^ are integral. Clearly 
t = t'g-g^^^-^ is the only point such that 7t'oq H 7^ 0, and 
again by applying Lemma 2.10 we may see that T^oQ is reduced. 

Case II Suppose Q = [A, B, C] is such that D < AA^, but neither 
p_i nor p+i hes in 7q. Clearly 7t*oq H JF'"* ^ 0. Now using the 
Lemma 2.10 we may see that I{'^t*oQ, c) = 1 for suitable t e {—1, 0, 1}. 
The corollary follows. □ 

Lemma 2.12. Suppose Q is an indefinite form such that either p_i e 
7q or p+i e 7q. Then -fq n .F*"* = i/ and only if ^soQ n JT™* 7^ 0. 

Proof. On the one hand note that the assumption p±i e 7q is equiva- 
lent to 

(2.2) 2{A + C) = tB, 

where Q — [A, B, C] . On the other hand note 7Qn J^*"* = is equivalent 
to 

(2.3) -^e(0,±l). 

Using Equation 2.2 we may see Inclusion 2.3 is equivalent to 

5 e (0,-1). 
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An immediate consequence of the latter is that the form 

Q' ^SoQ^[C,-B,A] 

satisfies 7q/ fl JF*"* 7^ 0. The converse is clearly true and the lemma 
follows. □ 

Definition 2.13. We say an indefinite form Q is normalisable if there 
is S{Q) e Z such that T^^^^ o Q is reduced. Such form Q""" = T^^^^ o Q 
is called its normalisation. 

Definition 2.14. The tip 7 of a geodesic 7 C 7i is the point 

X - y 



^ X + y 
7 = 



I E 7 



where 7 = {x,y} and x and y in M. We define the tip Q of an indefinite 
form Q as the positive definite binary quadratic form P such that 
T^p — iQ- If <5 is integral we assume Q is primitive. 

We will find it convenient to consider pointed spaces (7Q,ro), with 
To G 7(3, e.g. To = 7q. By a slight abuse of notation we sometimes 
write ((5,To) instead of (7q,to). 



Algorithm 2: The reduction of Q 
Data: (Q, tq) with ro7Q 
Result: reduced form Q^^"^ equivalent to Q 

1 Q < — Rto°Q; 

2 if 7q n J^'"* ^ then 

3 Q< — Q""" 

4 else 

5 Qi — {So Q)^^ 

6 end 



Now we prove that Algorithm 2 is correct. First note that the cor- 
rectness of Algorithm 1 (due to GauB) implies that after Step 1 the 
geodesic and the (closed) fundamental region share at least one 
point. If also the interior .F'"* and 7q have non-trivial intersection, 
then Corollary 2.11 implies that the normalisation Q"™ of Q is re- 
duced. However, it may sometimes happen that JF*"* n 7q = 0, which 
means that jq contains one of the lower vertices of the fundamental re- 
gion J^. But Lemma 2.12 tells us that the transformed form SoQ now 
belongs to the above case, and thus {S o Q)""" is reduced. Therefore 
Algorithm 2 is correct. 
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2.3. Regular path attached to an indefinite form. As before let 
Q — [A, B, C] be primitive indefinite form with non-square discrimi- 
nant D ^ B^ - AAC and define 

where (x, y) e Z x Z is the fundamental solution to Pell's equation 
(2.4) - DY"^ = 1 

such that both, x > and y > 0, so that the eigenvalue \q = x + 
y\fD e of Mq satisfies Aq > 1. Note that Mq is an automorphism 
of the QTiadratic form Q. So given a base-point tq G 7q, the geodesic 
segment {tq, MqTq) is a subset of the geodesic 

I 2A ' 2A J ■ 

Now, by standard results of hyperbolic geometry as in Beardon's book [2] , 
we may see that the arc-length parametrisation of the (oriented) geo- 
desic segment {tq, MqTq} is the restriction of the arc-length parametri- 
sation of the (oriented) geodesic jq with base point tq to the closed 
interval [0,/(Q)] C M, where 1{Q) = 21og(AQ) is the hyperbolic length 
of {tcMqTo}. Now let 

[0,/(Q)]— .X(l). 

be the parametrisation induced in the quotient X{1) by the above 
restriction. Note that (7(5)r(i) = {tq, MQTo}r(i), where -'^r(i) denotes 
the image of a subset X gH* in the quotient X{1) — r{l)\'H*. Since 
Q is primitive, the assumption that (x, y) is a fundamental solution to 
Equation 2.4 is equivalent to the assertion that Mq is not a non-trivial 
power of an element in the modular group r(l). So essentially Qq^to 
traces out its image (7Q)r(i) = {to, MqTo}y{i) only once. We caU gq^r 
the closed geodesic associated to the form Q and the point tq G 7q.^ 

Let X be any different iable manifold and let T be a positive real 
number. A diffcrcntiablc path g : [0, T] — > X is called regular if the 
derivative g'{t) 7^ 0, for each t E [0, T]. 

Lemma 2.15. Again let Q be an indefinite form and r a point of 'Jq. 
If the image of the path jq G H contains an elliptic point, then the 
path gq^T gH is not regular, else the path gq^^ gH is regular. 

Proof. It is well-known that the complex-analytic structure of the affine 
modular curve Y{1) — r{l)\H is induced by the map j: Ti ^ C, where 

^In the literature the closed geodesic (/q,to is also known as a prime geodesic. By 
the Prime Geodesic Theorem of Sarnak [7] we know that the distribution of the 
lengths 1{Q) is similar to the distribution of prime numbers. 
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j(r) is the elliptic modular function 

7(r) = 1 + 744 + 196884g + . . . , 

q 

(See Knapp's book [5].) Moreover, given any point h E Ti there is a 
sufficiently small neighbourhood U (resp. V) of b (resp. j{h)) together 
with an holomorphic identification 0: U — > D (resp. ■0: V — > D) 
with the unit disk 

D = e C: 1^1 < 1} 

such that the diagram 

3 




Z I > 

commutes, where 

{3, if h is an elliptic point of order 3, 
2, if b is an elliptic point of order 2, 
1, otherwise. 

So there is ^ [0) ^(Q)] such that (/Q,ro(^o) is an elliptic point then if 
and only if the derivative gQ^^{to) = 0. The lemma follows. □ 

For the rest of the subsection suppose Qq^to fails to be regular at a 
point to G [0,T], i.e. (?Q.r(,(^o) = where is (the image in X(l) of) 
an elUptic point of order e = 2 or 3. Shortly we shall see how, under 
certain mild conditions, a suitable restriction of the path qq^to may be 
extended in a natural way to produce an analytic regular path. But 
first we need to prove the following two elementary lemmas on square 
roots of certain elements of the group SL2(M), and also introduce a 
basic definition immediately after the lemmas. The mentioned regular 
path is actually constructed in the proof of Theorem 2.19. 

Lemma 2.16. LetQ and Mq be as above. Also lete — x'+y'\/D e 
(with x' and y' in \X) be a fundamental unit of the real quadratic order 
Od of discriminant D. Assume that e has negative norm A/'(e) = —1 
and also that e > 1. Then 

"^'TBK 2Ax' Dy' + Bx')^ ^^^^^^ 
is a square root of Mq such that the arc-length parametrisation of 

the geodesic segment {tq^ A{qTq} is the restriction of the arc-length 
parametrisation of the geodesic segment {tq, MqTq} to the interval [0, |/((5)]. 
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Proof. Consider the matrix 

/ -b+Vd „ -B-VD j\ 

where c and d are integers such that cd = It is plain that the 

matrix M maps the elements and oo as follows: 

-b-Vd 

0^ ^ / , 

2A 

-B + y/D 

2A 

Now recall that the solution (x, ?/) e Z x Z of Equation 2.4 is minimal 
subject to the conditions a; > and y > 0. In particular the eigenvalue 
Xq — x + yy/D of Mq is the generator of the cyclic subgroup 

{zeO^: N{z) ^landz> 1}. 



Now consider that the fundamental unit e = x' -\- y'yD has negative 
norm A/'(e) = —1. Thus = Xq (since both, Xq > 1 and e > 1). Hence 
M ( ^ ) Af-^ e SL2(i^) is a square root of Mq. It is plain that 

and the equality in the lemma follows. Now using (once more) that 
e > 1, the latter part of the lemma follows. □ 

Lemma 2.17. The matrix 

1/1 1 



52 - 

is the square root of S — {^il); it fixes i and induces the z ^ iz map 
in the tangent space ofH at i. 

Proof. The lemma follows after a couple of straight forward calcula- 
tions. □ 

Definition 2.18. Let X be a topological space and let Ti and T2 be 

positive real numbers. Suppose we have paths /i : [0, Ti] — > X and 
f2- [0,7'2] — > X such that fiiTi) = /2(0). The concatenation fi * /2 
of fi and /2 is the path 

/i*/2: [0,T, + T2]^X, 

where 

flit) tf^<t<Ti, 



Ui*h){t) = 

for each t e [0, Ti + T2] 



f2it-T,) tfT^<t<T^+T2, 
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Theorem 2.19. Assume that the base point tq of gq^ro is not an elliptic 
point of order e — 2. Also, if Qq^tq contains an elliptic point of degree 
e = 2, further assume that the fundamental unit e of Od has norm 
M{e) = —1. Then there is a regular analytic path pQ,^, on X{1) such 
that, up to a continuous reparametrisation of gq^To, the restrictions of 
pq^ro (^nd gq^ro to I ^[0, to) coincide 

for to G M>o sufficiently small. Moreover, up to a regular (analytic) 
reparametrisation of pq^ro, the path pq^ro is uniquely determined by 
9q,to ■ 

Proof. Suppose 7q contains an elliptic point Tg of order e — 3. By 
the commutative diagram in the proof of Lemma 2.15 we may see that 
for a small enough open neighbourhood U of Tg there is a continuous 
reparametrisation of the restriction gq^rol^ of 9q,to to the preimage 
V — gq\U, such that gq^rol^ is both, analytic and regular. Now 
suppose 7q contains an elliptic point Tg of order e — 2. Without loss 
of generality we may assume Tg = -\/— T £ H. The proof splits in two 
steps as follows. 

Step 1. We claim that the path gq^ri, may be decomposed as 

9q,to = {tq, b}r{i) * {b, ro}r(i) * {ro, b'}r{i) * {b' , To}r(i), 

for suitably chosen elliptic points b and b' in Ti. of order 2. Indeed, 

put b = Tg and b' = Mq ^ b. Now it suffices to prove that b' is an 

imaginary quadratic number of discriminant A = —4. If we consider 

the positive definite form P — [1,0,1], our claim follows if we prove 
_ i_ 

that P' — Mq ^ oP is integral, primitive, and has discriminant A = —4. 
Prom Definition 2.1 and Lemma 2.16 we get 

P' = [1 + 2x'^ - 2Bxy, 8Axy, l + 2x'^ + 2Bxy], 

where x + yy/D e is a fundamental unit (of negative norm) of the 
real quadratic order Od, and D is the discriminant of Q. So x and 
y both lie in |Z. (By a slight abuse of notation we have written x 

and y instead of x' and y'.) In particular we may see P' is integral. 

_i 

Now suppose P' is not primitive. So Mq ^ o P' = ( J "q^ ) o P is not 
primitive, but P — [1,0,1] is obviously primitive. We have reached a 
contradiction. Therefore P' must be primitive. Since it is plain that 
P' has discriminant A = —4, our claim follows. A consequence of our 
claim is that the path gq^ra not only fails to be regular at the points b 
and b'; these points are cusps (in the sense of differential geometry) of 
the path gq^^^. 

Step 2. Now we "remove" the cusps of gq^ro by replacing in the above 
decomposition the components 

{b, ro}r{i) * {to, &'}r(i) * {b', To}r(i) 
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To 




Figure 2.2. The path {tq, b} * {b, b'} * {b", tq}. 



by a certain concatenation of paths that meet at an angle of ^ at 
eUiptic points of degree e = 2. More precisely, we claim that a suitable 
continuous reparametrisation of the concatenation 

P = {ro, 6}r(i) * {b, 6'}r(i) * {b" , ro}r(i), 

1 

is the desired regular path, where b' — MJo b with a — sign{A), and 
_ 1 

b" — Mg^b. Since we assumed that Tq ^ b, the length to = /(tq, b) > 0. 
So = gQ^ro\I, where / = [0, to). Again by the commutative diagram 
in the proof of Lemma 2.15, the fact that the ramification degree e = 2 
implies that the path p is both, regular and analytic at the point b, after 
a continuous reparametrisation of p. Clearly the same is true for p at 
b' (considering b' and b" are the same point in X{1)). Denote pq^to 
such continuous reparametrisation of p. It is clear that pq^ro is unique 
up to a regular analytic parametrisation and the theorem follows. □ 

Example 2.20. Consider the indefinite form Q — [1,4,-1] of dis- 
criminant D = 20, and let Tq be the imaginary quadratic number 
To = G 7q. Clearly b = i, b' = i — A, and b" = i + 1, as 

depicted in Figure 2.2. Note the restriction gg^rol^ of 9q,to to / = [0, T) 
is regular, where T = 1{{to, b}) = 1.216185939 - • • > 0.^ Note gq^^o fails 
to be regular at b. Moreover, the path gq^^.^ clearly has a cusp (in the 
sense of differential geometry) at b. Since the geodesies jq and 
meet at an angle of |, and locally the function j(r) at b is the map 



^It is a curious fact that 



e'({To,b)} 



'218-|-96\/5 



V 38 

which obviously lies in a quadratic extension of the field 
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2; H- > z^, it is plain that the path {b',b} * {b, b"} is analytic and regular 
at b, if {6', b} * {6, b"} is suitably reparametrised. 

Remark 2.21. As before suppose that 7q contains an elliptic point Te 
of order e = 2. Note that the path qq^to is generically 2 : 1 onto its 
image, whereas the path pq^ro is generically 1 : 1 onto its image. This 
explains the italics in the word "essentially" by the beginning of this 
subsection, when describing how gg^ro traces out its image. 

2.4. Geometric coding algorithm I. Let Q be an indefinite form 
and let pg^ro be the closed regular path we attached to Q and Tq G •Jq 
in the above subsection. Here our main result is Algorithm 3, which 
produces the (geometric) code of pq^t^ with respect to the fundamental 
region of the modular group r(l). So our algorithm may be regarded 
as a variant on the geometric coding algorithm of Katok [4] . 
We need to introduce some preliminaries and notations. Let 

if A + C = 0, where a = sign{A) 
Sq=-{ 5±i, if2{A + C) = ±B^0 
otherwise, 

where 

s=(^\ J)' ^^^={^1 ±1)' 1 




Note 



^ , 1 ±1 1 
= - I t2 ±2 I , 
1 Tl 1 



5±2 2 



where M 1— > is the homomorphism defined by the beginning of this 
chapter. So we have 



S^-^ o [A, B, -A] 



In particular, S^^ o Q may be non-primitive — and even non-integral. 
Given any form Q with coefficients in Q, we define vq e Q>o to be 
such that vqQ is integral and primitive. 

Definition 2.22. If (Q) = (Qo, <3i, • • • Qz-i) is the cycle of {Q^Tq) 
produced by Algorithm 3, the code of the path pq^ro is the sequence 
of matrices (Mq, Mi, . . . , Mi_i) such that Qn+i = o Qn, obtained 
while performing Algorithm 3. 
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Algorithm 3: The cycle of Q 
Data: a pair (Q, tq) with ro7Q as above 
Result: the cycle [Q) = {Qo, Qi, • • • , Qi) of Q 

1 Q ^ Q'"^; 

2 Qo< — Q; 

3 n < — 0; 

4 while Q 7^ Qn or n = do 

5 Qn+1 ^ {^Qu^Qu ° Qn)"""; 

6 n n + 1; 

7 end 



Example 2.23. Consider the reduced form Q = [1, 12, —1] of discrim- 
inant D = 2'^ ■ 37. The cycle and code of Q are as follows. 

n Qn Mn 

[1,12,-1] (_^?) 

1 [3,5,-1] (5^) 

2 [-1,5,3] (YV) 

3 [3,1,-3] 

Note there is a "jump" in the discriminant after we encounter a sym- 
metric form. This is a consequence of the fact that the path 7q contains 
an elliptic point of order e = 2. 

Again let Q be an indefinite form of discriminant D (which we assume 
non-square). It is seems likely that the condition M{e) = — 1 on the 
fundamental unit e of O/j in our construction of the regular path Pq^tq 
may be dropped. In fact. Algorithm 3 makes perfect sense for any 
indefinite form Q (with non-square, positive discriminant). So one 
possible approach might be to (1) define Pq^tq as the image in A(l) of 
a path constructed in some way out of the code (Mq, Mi, • • • , Mi_i) of 
{Qi '^o) produced with the help of Algorithm 3, and then (2) prove that 
p'q (after a suitable continuous rcparamctrisation) has the desired 
properties. But we have not explored this approach any further in this 
dissertation. (In the applications we have in mind next chapter it is 
possible to prove that M{e) = — 1.) 

Example 2.24. Consider the reduced form Q = [5, 9, —7] of discrimi- 
nant D — 221. The cycle of the form Q is as follows. 



n 


Qn 


Mn 





[5,9,-7] 


(IV) 


1 


[-7,5,7] 




2 


[-5,32,-7] 




3 


[-7,24,11] 


(f V) 


4 


[11,20,-11] 





It is well-known that the fundamental unit of Od has positive norm. 
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3. Higher levels 

3.1. Reduction algorithms. Let F be a group acting on a Riemann 
surface X. A fundamental domain for F is a connected open subset D 
of X such that the following two properties hold. 

FD 1: No two points on D lie in the same orbit of F. 
FD 2: The closure D^''" of D contains at least one element from 
each orbit. 

The fundamental region associated to a fundamental domain D is the 
closure D'^^° of D. A fundamental set may be defined as a set D C C 
j^cio g^j.jj ^Yiai the canonical map from S into r\X is a bijection. 

Definition 3.1. Suppose 5* is a fundamental set for the action of F°(iV) 
on H. We say a positive definite form Q is S -reduced if ttq G J^n- 

We shall now describe a fundamental set for F°( AT) . As before assume 
that N is an odd prime. Define 

^(;V)^{-^.^.-2.-l}u{l.2.....^} 

and let T = ( J } ) and S = ( 5 V ) the standard generators of SL2(Z). 
It is well-known that the union 

jr^ = {SJ^''^"} U {T'J^"^": r e n{N)} , 

is a fundamental region for F*^(A^) with an even number of sides which 
are identified by the generators of F°(A^) 

= (J f ) , and M, = "^^^ ^) , e F^N), 

with s,s' e 71{N) such that r'r = —1 (mod N). (See e.g. Apostol [1], 
p. 76). 

Now we briefly review a construction of a fundamental set such 
that jFjv = {J^'j^Y''"^- First note that for each point r G (?(jFjv) = 
Tfq — T'f}^ the orbit F°(A^)t consists of at most 3 points, say, ri, r2, and 
Tj,. Clearly 3?(ri) 7^ ^iji) ii i 7^ j, for each i,j = 1,2,3. So it makes 
sense to define the left most point 1{t) of the orbit F°(Ar)T as the point 
Ti with the least real part 3?(Z(t)) < 3?(Tj), i — 1,2,3. Then we may 
define T'j^ as 

^;^ = ^-*U{/(r) :rGa(^iv)} 
which is by construction a fundamental set for F°(A^) such that J-'j^ — 

( 'T:! \ clos 

l-^ n) 

So given any positive definite form Q there is an element Rq^n £ 
F°(A^) such that Rq^n o Q is jF/y-reduccd. Such element may be com- 
puted as follows. Using Algorithm 1 wc may find a matrix Mi G 
SL2(Z) such that Q is .F-reduced. Now by trial and error we may 
pick a matrix M2 among the elements of the (finite) set of coset rep- 
resentatives {S} U {T": n G Tl{N)} of F°(Ar) in SL2(Z) such that 
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N 




R 







Figure 3.1. Fundamental domain for r°(iV), where 
Ar= 13. 

M — M2M1 e r*^(A^). Note that the imaginary quadratic r = ttq/ 
attached to Q' — M o Q clearly lies in the fundamental region Tn- 
If r e .F^* then Rq^n — M has required property. So we suppose 
r G d{TN)- If T ^ T'^ then for suitable choice among the generators 
G of r''(A^) introduced above we have Gt lies in the fundamental set 
JF^. So Rq^n = GM has the required property and we are done. 

Now we turn to the D > case. Let Q — [A, B, C] be an in- 
definite binary quadratic form of discriminant D = — 4AC > 0. 
As before we assume D is non-square, to avoid forms Q such that 
the geodesic •jq contains a side of the fundamental region J^n, e.g. 

Q = [1, 2, 0], [1, 4, 3], [1, 6, 8], Here we discuss an extension of the 

definition of normahsation and reduction of Q for the group r^(N). 

Lemma 3.2. Pick a base point tq of ^q, e.g. tq = Tq. The form Q is 
r'^{N) -equivalent to a form, we denote Q™*; such that ^qmt meets .F^* 



Proof. Now let Rtq,n denote the matrix in r''(A^) such that Tq = Rto,n° 
To is .^AT-reduced. Clearly the geodesic "Jq^ attached to Qi — Rj^^n ° Q 
is such that 



the point Tq is one of the non-cusp vertices of the fundamental region 
J-'n- So let us suppose p±i ± r G Jq^ and n J^^'* = 0.^ Put 

^This case occurs very seldom in practice. For example, fix 6 e M>o moderately 
large, and also fix a positive (non-square) discriminant D. Experimental evidence 
suggests that the probability that a form Q = [A, B, C] picked randomly in the set 

{Q = [A, B, C] e : \A\ < b, \B\ < b, \C\ <b,B'^ - AAC = D, and Q primitive} 




j^}, i.e. when 
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Q2 = T * o where s = ±r and note that contains the point 
p±i. Now by Lemma 2.12 we know that 

Qs = SoQ^ = [C2,-B2,A2] 

is such that 7^3 has non-trivial intersection with the open set 
Therefore Q4 — T^' o is such that has non-trivial intersection 
with T^':F'"* c J^P\ for each s' e n(N). In particular if ss' = -1 
(mod N) then the form 

Q4 = (T^'ST-^) o Q, = (T^'ST-^Rq ,,) o Q 

is clearly r°(A^)-equivalent to Q, and it is plain that 7Q4 n JF^* 7^ 0. □ 

Given a geodesic segment 7 = {a,P} let 7™* be the open geodesic 
segment obtained by removing the endpoints a and /3 of 7. So let 
c±i,oo C 5(^iv) be the open geodesic segment defined by 

r AT - 1 ^ 



2 



AT- 1 



int 



(^+1,00 = i p+1 H ^ — ,00 

In a similar way let us define the open geodesic segment a±i q C d{Tj^) 

by 

(7_i,o = {/9-l,0}'"* 

cr+1,0 = {0,p+i}'"* 

Now for the rest of this paragraph let us restrict the discussion only to 
forms Q such that 7q fl J^^* 7^ 0. We say Q is cZose to ^/le cusp 00 if 
I {1 Q J (^sign{A), 00) — 1- This is equivalent to the condition 

3|A| + sign{A){AC + N'^A) + 2ArB < 0. 

The N -normalisation Q^-""" of a form Q close to the cusp oo is the 
r°(iV)-equivalent form Q^-""" = t[t^]^ o g, where [•] is the nearest 
integer function. We say Q is close to the cusp if /(7q, Csi5n(A),o) — 1- 
This means that 

3|C| + sign{C){AA + C)-2B<0. 

The N -normalisation Q^-"™ of a form Q close to the cusp is the 
rO(iV)-equivalent form = {ST^^^°^^S) o Q, where S is as in 

Definition 2.13. For completeness, if Q is neither close to the cusp 
oo nor close to the cusp we define the N -normalisation Q^-""" of 
Q to be Q itself. Finally, the N- reduction Q^''^ of an arbitrarily 
given indefinite form Q (with non-square discriminant) is the normal- 
isation Q^-''^'^ = [Qrni)N-nrm Qmt ^ ^^^^^ Qmt jg ^j^g ^^^^ defined in 



falls in this case is small. 
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Figure 3.2. The AT-reduction oi Q = [-13, 108, -213], 
with N = 13. 

Lemma 3.2. If a form Qi is the A^-reduction of a form Qq then we say 
Qi is N-reduced. 

Example 3.3. Let = 13 and consider the primitive indefinite form 

Q = [-13,108,-213] 

of discriminant D = 588. Note 7(5 HJF^* = 0. Also p+i + r G 7q for r = 
4. So by the proof of Lemma 3.2 we may see that Q"** = {T^ST'^) o Q 
is a r°(A'")-equivalent form such that 7Qmt meets the open set J^J^^. Now 
since Q™* is neither close to the cusp oo nor close to the cusp 0, then 
by definition Q^'^'^'^ = Q"^\ as depicted in Figure 3.2. 

3.2. Geometric coding algorithm II. Note that rO(Ar) = 5-iro(Ar)5, 

where S — By a slight abuse of notation^ from now on we write 

Xo{N) = r°{N)\n*. 

As before let Q = [^, -B, C] be primitive indefinite form with non-square 
discriminant D ^ B"^ - AAC. Now define Mjv,q = M^, where 

fu — Bv —2Cv\ 
^<^=[2Av « + S^;J^^(l), 

and {u, e Z is a fundamental solution of the ordinary Pell equation 

- DX^ = 1, 

and n is the smallest positive integer such that Mq e ro(A^). Now 
pick a base point Tq of 7q and let gN;Q,To denote the image of the path 

{ro,M7v,QTo} in the modular curve Xq{N). The construction of the 
regular path in the proof of Theorem 2.19 carries over to Xo(A^) as 
follows. It is well-known that the elliptic points Tg G for the action 

'''it is more customary to write X°{N) = T°{N)\H*. But we want to avoid writing 
non-standard expressions such as X^{N) (or X^~^{N)) when considering the Atkin- 
Lehner quotient defined by the Pricke involution wn- 
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of r''(A^) in the upper half plane 7i are precisely the ones attached to 
primitive forms P — [A, B , C] with N \ C and discriminant A = — 3, if 
the degree of the point is e = 3 and discriminant A = —4, if the degree 
of the point is e = 2. Assume there is an elliptic point Te of degree 
e = 2, e.g. Tg = r + e Ti with r e Z such that 

r^ = -l (modA^), 



and that 7q contains such elliptic point Te- The matrix 5" 2 in the 

. 1 )• 



said construction is to be replaced by T^S'^T ^, where and T — ^ ^ ^ 



Again let n be the smallest positive integer such that Mq lies in r°(A). 

Further assume that the fundamental unit e of the real quadratic order 

1 1 

Od has norm A/"(e) = — 1. The matrix Mq is to be replaced by M^q = 
1 i_ 

(Mq)", so that {M^qY = Mn^q. Let Pn,q,to be the regular analytic 
path thus obtained and call it the regular path of level N attached to 
Q. This path is unique up to a regular analjd;ic reparametrisation. 

Now we need to extend Definition 2.13 to include some indefinite 
forms Q that arise in Algorithm 4. As before put a = {p-i, p+i}- 
Let 6{Q) be just as in Definition 2.13, ii D > AA^ and let 6{Q) be the 
integer such that the intersection number I{'yQ+S(Q),a) = 1 (whenever 
it exists), if SA^D < AA^. Assume that Q is A/'-reduced and that Q 
is neither close to cusp 00 nor close to cusp 0. Now put a — sign{A), 
d = S{Q), and define the matrix 



(3.1) Sn,q = < 



T'^ST-'^, if d = Q,i.e. Pa^^Q (1) 

T-"^5T-2'', ifp^ejQ-a (2) 

j^id-argj^d-a^ if p^e^Q + d and\d\y^l (3) 

T'*S^T'^, ifiN^lQ (4) 

T'^'ST^, otherwise, (5) 



where in = i + k and k e C{N) is such that k'^ = —1 (mod N), 
whenever A" = 1 (mod 4), and s* is the element of C{N) such that 
ss* = 1 (mod N), given s e Z with s ^ (mod A). 

Now we shall consider some examples of A^-cycles (Q) jv = {Qo, ■ ■ ■ , Qi-i) 
that may help to clarify the above ideas. In the tables below the last 
column encodes the steps followed while executing the algorithm, using 
the the convention that a number from 1 to 5 indicates one of the five 
cases of the definition of matrix Sn,q, while (resp. 00) indicates the 
normalisation step associated to a form close to the cusp (resp. the 
cusp 00). 



Example 3.4. Suppose A" = 13. Consider the indefinite form Q — 
[-13, 108, -213] of discriminant D ^ 7^ ■ 12. The A^-reduction of 
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Algorithm 4: The A^-cycle of Q 
Data: a primitive indefinite binary quadratic form Q 
Result: the A^-cycle {Q)n = {Qo, Qi, ■ ■ ■ , Qi-i) of Q 

1 Q ^ Q""-^"; 

2 Qo< — Q; 

3 n < — 0; 

4 while Q ^ Qn or n = do 

5 if Qn is either near cusp oo or cusp then 

« /O , r^N-nrm 

6 < Vn 

7 else 

8 (5n+l ^QnSN,Qn ° Qn 

9 end 

10 n i — n+ 1; 

11 end 



QN-red ^ ^11^ _7Q^ g ^las the following AT-cycle. 



n 


Qn 




Case 





[11,-70,98] 


il--'!) 


5 


1 


[-6,18,11] 


(}?) 





2 


[-13,-4,11] 


(}?) 


5 


3 


[2,-26,11] 


a-F) 


oo 


4 


[2,26,11] 


(}?) 


5 


5 


[-13,4,11] 


(}?) 





6 


[-6,-18,11] 


i-m 


5 


7 


[11,70,98] 


{-m 


3 


8 


[2, -2, -73] 




3 


9 


[11,18,-6] 







10 


[11,-18,-6] 




2 


11 


[2,2,-73] 




3 



Figure 3.3 depicts the intersection of the geodesies 7q„, with the funda- 
mental domain J^^, for n = 0, 1, 2 and 3. Similarly, Figure 3.4 depicts 
the intersection of the geodesies 7q„, with the fundamental domain T^, 
for n = 9, 10 and 11. Note that instance (4) of Sn^q in Equation 3.1 
does not take place. This means that the geodesic jq does not contain 
an elliptic point of degree e = 2. 
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Figure 3.3. Beginning of the iV-cycle of Q = 
[-13, 108, -213], with N = 13. 




Figure 3.4. Near the end of the A'"-cycle of Q = 
[-13, 108, -213], with = 13. 

Example 3.5. Now suppose = 5 and consider the A^-reduced form 
Q — [1, —1, —3] of discriminant D — 13. The TV-cycle of Q is as follows. 



n 


Qn 




Case 





[1,-1,-3] 


i'-ll) 


4 


1 


[3,-16,17] 




oo 


2 


[3,14,12] 


(}?) 


3 


3 


[1,-10,12] 




oo 


4 


[1,10,12] 


(}?) 


3 


5 


[3,-14,12] 




oo 


6 


[3,16,17] 


(-\-\) 


4 


7 


[1,1,-3] 




5 


8 


[-1,-5,-3] 


ill) 


oo 


9 


[-1,5,-3] 




5 
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Qo • • • (4) 




Figure 3.5. Beginning of the A/"-cycle of Q = 
[1,-1,-3], with TV = 5. 

Figure 3.5 depicts the intersections of the geodesies '^q^ with the fun- 
damental domain !Fj^, for n — 1,2, and 3. Note that Qn falls in case 
(4), for n = and 6. This means that 7q contains an elliptic point Tg 
of order e — 2. 

Remark 3.6. It is not hard to see that for each prime > 1, the 
A'^-cycle of the the indefinite form Q — \2, 2N, —1] is given by 

n Qn Mn 

[2,2iV,-l] 

1 [2,-2iV,-l] (i-f) 

Note that the matrices Mq = ( 5 ) M) = ( o ) parabolic. 
So the code (Mq, Mi, . . . ) of some indefinite forms Q may consist of 
parabolic elements only. 

Definition 3.7. The N-code of Q is the sequence of matrices (Mq, Mi, ... ) 
defined by Qn+i — M„o(5„, where the matrices M„ come from the com- 
putation of the iV-cycle {Q)n — {Qo,Qi, ■ ■ - Qi) oiQ using Algorithm 4. 
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